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ABSTRACT 

This study explores the effect of user and system characteristics on our proposed web-based classroom response 
system (CRS) by a longitudinal design. The results of research are expected to understand the important factors 
of user and system characteristics in the web-based CRS. The proposed system can supply interactive teaching 
contents, quizzes, and questions from its database. In addition, the system provides three playful classroom 
response modes—the individual assigned mode, the rush answering mode, and the pushing button mode—to 
increase computer playfulness, friendly interface, and interactivity for students. Both user and system 
characteristics are based on the external variables from the technology acceptance model (TAM). The results of 
the longitudinal empirical study indicate that the effect of user characteristic on perceived usefulness is more 
than that of system characteristic, while the effect of system characteristic on perceived ease of use is more than 
that of user characteristic. Furthermore, the difference between short-term use and long-term use is analyzed. In 
the user characteristic, the effect of long-term use on perceived usefulness is more significant than that of 
short-term use. The effect of system characteristic on perceived usefulness is insignificant for long-term use, but 
that is significant for short-term use. The effect of long-term use on perceived ease of use is more significant 
than that of short-term use. In addition, computer playfulness is positively related to perceived ease of use, while 
subjective norm affects directly on perceived usefulness. 

INTRODUCTION 

Classroom response systems (CRSs) have been widely used to assist students in learning. CRSs, capable of 
capturing student votes and transmitting data via communication networks, are designed to collect and aggregate 
student responses instantly, and display the aggregated results in the class. However, students are asked to 
purchase radio frequency key pads to communicate in both directions, thus creating additional technology cost 
for schools and students. Therefore, although some universities adopt CRSs to assist teaching activities, others 
rarely use it. Logistical difficulties for teachers and added cost to students mean that CRSs must have a clear 
benefit and allow easy creation of new client applications. Online learning provides the perfect environment for 
web-based CRSs as all students have computer and access to the internet. Among general studies about CRSs 
use are reviews of CRS literature, general recommendations of use, and technical descriptions and comparisons. 
However, CRSs applied in the technology acceptance model (TAM) is rarely explored. 

TAM from information systems field has been widely applied in the use of educational information technology. 
The results of past studies (Chunga &Tan, 2004; Drave, 2000; Ha & James, 1998; Hasn & Ahmed, 2007; 
Hoffman & Novak, 1997) indicated that the variables of system characteristic could affect perceived usefulness 
and perceived ease of use. Although TAM was examined as a model to explain the influence of external 
variables on using e-leaming systems, these systems rarely included classroom response mechanism. Thus, we 
have developed an innovative web-based CRS using web services to create the system with computer 
playfulness, friendly interface, and interactivity that enhance the teaching and learning experience and remain 
extensible and developer friendly. 

According to TAM, perceptions of ease of use and usefulness are directly determined by external variables. 
Although these external variables pertain to user characteristic and system characteristic, most past studies have 
only focused on examining the role of individual characteristic of user or system for e-learning systems and 
information systems (Park, 2009; Liu, Liao, & Peng, 2005; Lee, Cho, Gay, Davidson, & Ingraffea, 2003; Ngai, 
Poon, & Chan, 2007). However, little attention has been given to examining the impacts of both user and system 
characteristics on web-based CRSs. In addition, the research design of most past studies adopted cross-sectional 
design to collect data at one point in time. It limited causal inferences because temporal priority was difficult to 
establish. When the problem is with the examination of a dynamic process that involves change over time, 
longitudinal design is the most appropriate. The questionnaires of our experiment in this study examined two 
points in time and are based on the longitudinal design. 
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The purpose of the study is to understand the effect of user and system characteristics on perceived usefulness, 
perceived ease of use, and attitude toward use for the web-based CRS. In addition, we want to explore the 
difference of students’ grades between the group using the system and the one not using the system. 

In the user characteristic of our research model, we examined three external variables—subjective norm, 
self-efficacy, and personal innovativeness—in the domain of information technology (PUT) to understand the 
effect of them on our proposed web-based CRS. Subjective norm from the theory of planned behavior provides 
positive attitude to reduce the cognitive and mental effort. Self-efficacy from the social cognitive theory is a 
more powerful direct determinant of ease of use. PUT, the willingness of an individual to try out any new IT, 
from the innovation diffusion theory plays an important role in determining the outcomes of user acceptance of 
technology. 

In the system characteristic, we built interactive teaching contents, quizzes, and questions banks. The system 
provides learning track function, synchronous classroom function, assignment management function, questions 
management function, and teaching material management function. In the proposed CRS, we kept most 
functions typical of CRSs and added playfulness, friendly interface, and interactivity of answering questions into 
the system. Therefore, we also examine three variables in the system characteristic such as computer playfulness, 
interface style, and interactivity. Computer playfulness, which affects user acceptance, is an intrinsic motivator, 
which can be described as the most enjoyable experience possible. Interface style can affect the perceptions of 
ease of use and of usefulness. Interactivity plays a crucial role in knowledge acquisition and the development of 
cognitive skills, and that interaction is intrinsic to effective instructional practice and individual discovery. 

The perceived usefulness and the perceived ease of use from TAM are determined by the theory of computer 
playfulness (Woszczynski, Roth, & Segars, 2002), the theory of planned behavior (Ajzen, 1991), the social 
cognitive theory (Bandura, 1997), the innovation diffusion theory (Rogers, 1995), and the system interface 
research (Hasn & Ahmed, 2007). Our proposed user and system characteristics are based on the aforementioned 
theories. Hence, our study provides a theoretical perspective on how user characteristic and system characteristic 
affect the intention of using the web-based CRS. In this paper, the contributions drawn are as follow: (1) A 
web-based CRS with computer playfulness, friendly interface, and interactivity is developed; (2) The impacts of 
both user and system characteristics for the web-based CRS are examined; (3) The longitudinal design is adopted 
for raising causal inferences; and, (4) Understanding the impact of system characteristic on users’ perception of 
the system provides valuable implications for practice and improving web-based CRS acceptance. 

LITERATURE REVIEW 

According to our research goal, we first surveyed classroom response systems. Second, we explored two 
constructs, user and system characteristics, of external variables in TAM. Finally, we examined past studies on 
the relationships between web-based learning systems and TAM. 

Classroom response systems 

A recent analysis of 24 classroom network studies by Fies & Marshall (2006) indicated that there was good 
agreement in terms of benefits of use. Specifically, they found that indications of greater student engagement 
increased student understanding of complex subject matter, student interest and enjoyment, student awareness of 
individual levels of comprehension, and teacher insight into student difficulties and heightened discussion and 
interactivity. Preszler, Dawe, Shuster, & Shuster (2007) assessed the effectiveness of wireless student response 
systems in the biology curriculum. Their results indicated that students had favorable opinions about the use of 
classroom response systems and that the increased use of these systems raised student learning. Seletsky (2009) 
made a web-based system that could easily be written from a client for many input devices. The system was 
added a confusion meter feature to allow teachers to gauge the appropriateness of the teaching pace without 
continuously asking questions. In addition, it was added a set of features that allowed students to work with 
questions outside the class as a way of review and further discussions. Lowery (2005) described the components 
and operation of the two most common types of student response systems, wireless keypad and web-based input 
devices. The web-based systems offered far more powerful student input devices than typical keypad systems. 
Moss and Crowley (2011) indicated that these systems provided a highly flexible and transferable approach to 
the use of interactive technology for engaging learners of all ages as well as carrying out research. Gok (2011) 
proposed an evaluation method for student response systems. The results indicated that these systems were 
especially valuable tool for introductory courses and for monitoring peer learning methods in large lecture 
classroom. However, almost all of the studies were based on typical question format that required all learners to 
select from given options. Beyond traditional CRSs, all students were asked to indicate whether a statement was 
true or false, or to select an option. Therefore, three rare modes were developed in our proposed web-based 
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classroom response system in order to improve traditional CRSs with computer playfulness, friendly interface, 
and interactivity. 

External variables in TAM 

There are three variables in user characteristic, including subjective norm, self-efficacy, and personal 
innovativeness in the domain of information technology (PUT). Subjective norm from the theory of planned 
behavior (Ajzen, 1991) has been defined as a person’s perception of whether most people who are important to 
him think he should or should not perform the behavior in question. Schepers & Wetzels (2007) conducted a 
quantitative meta-analysis of previous research on TAM in an attempt to make well-grounded statements on the 
role of subjective norm. Subjective norm was directly related to intention to use in most studies. Schepers & 
Wetzels (2006) thought when people were in individual environment, subjective norm was included in an effort 
to enhance the understanding of user’s adopted behavior. 

The social cognitive theory (Bandura, 1997) posited that people were neither driven by inner forces, nor simply 
by external stimuli. Instead, human behavior was explained via a model of triadic reciprocity in which behavior, 
cognitive and personal factors, and environmental events all operate interactively as determinants of each other. 
A key regulatory mechanism in this dynamic relationship that affected human behavior was self-efficacy, 
people’s judgments of their capabilities to perform a given task. Many empirical studies validated this 
proposition in a wide variety of settings. Hasan (2006) discussed extensions to previous research on computer 
self-efficacy (CSE) and systems acceptance by examining the impact of multilevel CSE on information system 
acceptance. The results of a field experiment indicated that system-specific CSE represented a stronger predictor 
of perceived usefulness and behavioral intention than general CSE. Prior research (Yi & Hwang, 2003) on 
technology acceptance behavior examined the effects of self-efficacy on ease of use. Goal orientation has been 
recognized as important in understanding individual differences in motivated behavior. The study demonstrated 
a more direct and powerful effect of application-specific self-efficacy on perceived ease of use. The results 
indicated that users regarded the system easier to use when their conviction in their own efficacy regarding the 
target system was higher and that application-specific self-efficacy was a more powerful, direct determinant of 
ease of use than general computer self-efficacy. Chen & Tseng (2012) indicated that the Internet self-efficacy of 
teachers with respect to using web-based e-leaming systems for in-service education had a significant positive 
influence on perceived usefulness and perceived ease of use. 

Personal innovativeness in the domain of information technology (PUT) (Rogers, 1995), the willingness of an 
individual to try out any new IT, played an important role in determining the outcomes of user acceptance of 
technology. Within the context of IT adoption, Brancheau & Wetherbe (1990) found that, relative to later 
adopters, earlier adopters were involved in significantly more interpersonal communication. The earlier adopters 
typically had the ability to envision the potential benefits and advantages associated with an innovation in its 
early stage of diffusion. Lewis, Agarwal, & Sambamurthy (2003) found that PUT was a significant determinant 
of perceived ease of use. Extending prior findings, we theorized a direct effect of PUT on perceived usefulness 
and perceived ease of use. 

The above three variables are based on user characteristic, reflecting the cognitive and mental effects of learners 
for perceived usefulness and perceived ease of use. Therefore, these variables assist in understanding the effect 
of user characteristic. 

For system characteristic, there are three variables, including computer playfulness, interface style, and 
interactivity. Chunga & Tan (2004) found that computer playfulness had a significant positive relationship with 
attitude toward use. They concluded that it was important for developers to include intrinsic and extrinsic 
motivational factors in perceived usefulness and perceived ease of use, thus helping to improve users’ attitude 
toward use. Sun & Cheng (2009) indicated that playfulness had a significant effect on perceived usefulness and 
ease of use. Shyu & Huang (2011) pointed out that perceived enjoyment positively affected perceived ease of 
use for adopting acceptance of government information system. Park et al. (2012) showed that enjoyment had a 
significant effect on perceived usefulness and perceived ease of use. 

Hasan & Ahmed (2007) examined the influence of two interface styles (menu- and command-based) on the 
perceived ease of use, perceived usefulness, and behavioral intention of the users to use office automatic systems. 
Tucker (2008) applied the TAM to predict user attitude toward using electronic commercial systems by 
conducting a web-based survey. The results indicated that the interface style of system design had a significant 
positive effect on perceived usefulness, and attitude toward use. Davis (1992) compared the impact of 
direct-manipulation and command based interfaces. Their results revealed that interface style had no significant 
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effect on ease of use. However, perceived usefulness was not examined in the study. Similar results were 
proposed by Gururajan & Fink (2002), who compared the impact of icons and menu interface styles. Sun & 
Cheng (2009) indicated that the factor of interface style had a significant effect on perceived usefulness and ease 
of use. 

Hoffman & Novak (1997) found that affecting the attractiveness of interactive systems was an important factor. 
Cawthon & Moere (2007) assessed the attractiveness of interactive systems. Dagiene & Futschek (2008) thought 
that the attractiveness of interactive tasks was important and the effort for implementing the interactive part was 
worth doing. Interactive multimedia is a strong feature and enables users a new environment that provides more 
services than traditional teaching. Several studies discussed interactivity in business websites and reported that it 
was useful for distance learning research and design. Ha & James (1998), in their baseline analysis of business 
websites’ interactivity, defined interactivity as the extent to which the communicator and audience respond to 
each other’s communication needs. Drave (2000) put together a list of interaction modes that included single 
thread asynchronous discussion forums, threaded bulletin boards, real-time synchronous chat rooms, and email. 
Siau, Sheng, & Nah (2006) examined the effects of a classroom response system on interactivity. The results all 
showed that a classroom response system could significantly improve classroom interactivity. 

The three variables mentioned above are based on system characteristic, playing important roles in determining 
the outcomes of user acceptance. 

Relationships between web-based learning systems and TAM 

Technology acceptance model (TAM) has constructed acceptance of a system as a function of users’ perceptions 
of the ease of use and usefulness of system (Davis, 1989). More importantly, it suggested that perceptions of 
ease of use and usefulness were directly determined by external variables. The influence of technology 
acceptance model on the use of e-leaming system has been widely investigated in recent literature (Lee et al., 
2003; Liu et al., 2005; Ngai et al., 2007; Park, 2009; Seal & Przasnyski, 2001; Sheremetov & Arenas, 2002). 
Seal & Przasnyski described an implementation of the continuous improvement philosophy in a graduate class 
by using web services to obtain immediate and systematic feedback from students on lectures and other course 
activities. The feedback obtained was analyzed to determine how the delivery and content of the course could be 
improved. Sheremetov & Arenas proposed a system based on web learning environment. All user interfaces 
were web pages, generated dynamically by servlets, applets, and agents. Park surveyed a sample of 628 
university students to figure out the students’ behavioral intentions to use typical e-leaming system. Liu et al. 
proposed an integrated theoretical framework for users’ behavior on web-based streaming e-leaming. The study 
considered the e-leaming systems user as both a system user and a learner. They designed three presentation 
types of e-learning course, including text-audio, audio-video, and text-audio-video to test their theory 
development. Lee et al. proposed a web-based application called advanced interactive discovery environment 
(AIDE). AIDE is a virtual environment containing application-specific content, application-appropriate 
simulation and software packages, distributed learning modules, expert systems, knowledge bases, and 
synchronous and asynchronous communication tools, including message boards, instant messaging, chat, and 
multi-point audio and video. Ngai et al. extended TAM to include technical support as a precursor and explored 
the role of extended model in user acceptance of web course tools. 

In addition, Liu et al. (2010) took the TAM as a foundation to discuss the effect of extended variables on 
whether users adopt an online learning community. Shyu and Huang (2011) explained and predicted usage of 
e-government learning based on the TAM to examine how perceived ease of use and perceived usefulness and 
their antecedents influenced intention and usage of a system. Arenas-Gaitan et al. (2011) examined cultural 
differences and technology acceptances from students of two universities, one Spain and Chile. The results 
indicated that students were culturally different with regard to some dimensions, but their behavior of acceptance 
of e-learning technology matched globally according to the TAM model. Park et al. (2012) employed an 
extension of the TAM to investigate factors that influenced successful implementation of a web-based training 
system in the construction industry. Chen and Tseng (2012) used the TAM as a foundation to add into teacher's 
perspective and examine factors that influence intentions to use in-service training conducted through web-based 
e-leaming. Lin (2012) integrated information system continuance theory with task-technology fit to extend the 
TAM for understanding the precedents of the intention to continue virtual learning system and their impacts on 
learning. 

However, most past studies have focused on examining the role of individual characteristics of users, and few 
researchers have explored the impacts of both user and system characteristics on CRSs, especially on web-based 
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CRSs. 

RESEARCH APPROACH 

In the theoretical fundamental, we adopted the theory of technology acceptance model (TAM). We constructed 
the conceptual model of TAM to assess the relationships between system characteristic, user characteristic, 
usefulness, ease of use, and attitude toward use, as shown in Figure 1. 

Over the last decade, substantial empirical evidence has accumulated to support TAM (Yi, Jackson, Park, & 
Probst, 2006). Attitude toward using a new system is determined by perceived ease of use and perceived 
usefulness. 



Figure 1. Research Model. 


In addition, intention to use is also determined by attitude toward use. Our research model is adapted from prior 
research. In this study, we want to understand the effect of user and system characteristics on perceived 
usefulness, perceived ease of use, and attitude toward use for our proposed web-based CRS. Therefore, we test 
the relationships among user characteristic, system characteristic, perceived ease of use, perceived usefulness, 
and attitude toward use with the following hypotheses Hi to H 9 . 

Hi. Perceived ease of use is positively related to perceived usefulness. 

H 2 . Perceived ease of use is positively related to attitude toward use. 

H 3 . Perceived usefulness is positively related to attitude toward use. 

User characteristic 

For the user characteristic, subjective norm represents the perception, by which way other people expect the 
person to behave. It is expected to play a significant role in forming their mental models. Therefore, the 
subjective norm that provides positive attitude to reduce cognitive and mental effort is expected to receive more 
favorable evaluations from users. We follow this trend and, consistent with existing research, hypothesize that 
subjective norm will have positive impacts on perceived usefulness, perceived ease of use, and attitude toward 
use. 

Self-efficacy plays an important role in determining a person’s behavior. Following similar reasoning, we 
hypothesize that self-efficacy is positively related to perceived usefulness, and attitude toward use. 

Earlier adopters of innovative technology would feel more confident about the tangible results. Therefore, we 
hypothesize that personal innovativeness in the domain of information technology (PUT) is an antecedent to 
perceived usefulness, perceived ease of use, and attitude toward use. Therefore, we test the relationships between 
user characteristic and perceived ease of use, perceived usefulness, and attitude toward use with hypotheses H 4 
to H 6 . 
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H 4 . User characteristic is positively related to perceived usefulness. 

H 4 -!. Subjective norm is positively related to perceived usefulness. 

H 4 - 2 . Self-efficacy is positively related to perceived usefulness. 

H 4 _ 3 . PUT is positively related to perceived usefulness. 

H 5 . User characteristic is positively related to perceived ease of use. 

H 5 _i. Subjective norm is positively related to attitude perceived ease of use. 

H 5 - 2 . Self-efficacy is positively related to perceived ease of use. 

H 5 - 3 . PUT is positively related to perceived ease of use. 

H 6 . User characteristic is positively related to attitude toward use. 

H 6 -!. Subjective norm is positively related to attitude toward use. 

H 6 - 2 . Self-efficacy is positively related to attitude toward use. 

H 6 _ 3 . PUT is positively related to attitude toward use. 

System characteristic 

Computer playfulness can affect user acceptance as an intrinsic motivator which can be described as the most 
enjoyable experience possible. Interface style of system design has strong effects on the determinants of system 
acceptance. Interactivity in web sites can be useful for distance learning. In our related work, the authors found 
that computer playfulness, interface style, and interactivity have a significant positive relationship with users’ 
perceived usefulness, perceived ease of use, and attitude toward use. Therefore, perceptions of system 
characteristic are influenced by user perceptions of computer playfulness, interface style, and interactivity. There 
are other system attributes, such as system reliability, documentation, and functionality that have the potential to 
influence users’ perceptions of usefulness and ease of use. We, however, chose to focus on the impact of 
computer playfulness, interface style, and interactivity for several reasons. 

First, there has been little research in this area and several researchers, as described above in the section of 
literature review, have called for additional research to gain better insights into the role of educational 
information technology. Second, given the mixed and inconsistent results, knowledge of how system 
characteristic affects perceptions of user remains incomplete. Third, from a user’s perspective, system 
characteristic represents a user’s perception of using the system, not system functionality and documentation. 
Finally, since the system characteristic is completely under the control of system designers and developers, 
reliability is a basic requirement. Therefore, we test the relationships between system characteristic and 
perceived ease of use, perceived usefulness, and attitude toward use with hypotheses H 7 to H 9 . 

H 7 . System characteristic is positively related to perceived usefulness. 

H 7 _!. Computer playfulness is positively related to perceived usefulness. 

H 7 _ 2 . Interface style is positively related to perceived usefulness. 

H 7 - 3 . Interactivity is positively related to perceived usefulness. 

H 8 . System characteristic is positively related to perceived ease of use. 

Hg-!. Computer playfulness is positively related to perceived ease of use. 

H 8 _ 2 . Interface style is positively related to perceived ease of use. 

H 8 _ 3 . Interactivity is positively related to perceived ease of use. 

H 9 . System characteristic is positively related to attitude toward use. 

H 9 _i. Computer playfulness is positively related to attitude toward use. 

H 9 _ 2 . Interface style is positively related to attitude toward use. 

H 9 _ 3 . Interactivity is positively related to attitude toward use. 

CRS-based vs. traditional learning performance 

To understand the difference between CRS-based and traditional learning performance, we selected two classes 
to experiment. One used web-based CRS materials, denoted as CRS-based group, while the other was taught in 
the conventional way, denoted as tradition-based group. Students of the two classes took a visualization 
programming course, a three-hour-per-week course lasting 18 weeks. The grades of CRS-based class are 
compared with those of traditional learning class. Therefore, we tested the difference between traditional and 
CRS-based learning performance with hypothesis H 10 . 

H 10 . The performance of CRS-based learning is significantly different from that of traditional learning. 
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Items used in the questionnaire to operationalize the constructs in our research model are adapted from 
prior research in our related work with changes in wording appropriate for the system in the targeted context. All 
of the items shown in appendix A are measured on a five-point Likert scale ranging from 1 {strongly disagree) to 
5 {strongly agree). 

The Proposed Web-based Classroom Response System 

We developed the web-based classroom response system to support instructional sequence and web-based 
instantaneous feedback aggregation. The architecture of the system consists of four main components: course 
entry, course management, test banks, and material management. The course entry provides component areas for 
assignment, bulletin, online classroom, learning track, reference material, and discussion. Figure 2 illustrates the 
main functions of course entry. 



Figure 2. Main functions of the course entry for the proposed web-based CRS. 



Figure 3. Online classroom of the proposed web-based CRS. 

The online classroom provides students with online users list, question responding, synchronous video, online 
conversation, and slide display. Figure 3 illustrates the situation of online teaching. In the online users list, 
instructor can assign questions from test banks to online students and ask students to respond to the questions in 
“question responding”. This question function has four modes for instructors to choose from. The first mode, 
typical question, is used to ask all online students to answer the question. The responding student is asked to 
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indicate whether a statement is true or false, or to choose an option among given choices predetermined by the 
instructor. Figure 4 illustrates the situation of the typical question mode. The system can provide reliable 
aggregation of students’ responses with interpretable output in the form of a PI graph and keep the learning 
track. 



Figure 4. Typical question of the proposed web-based CRS. 

In addition, three playful modes have been developed in this web-based classroom response system to raise 
computer playfulness, friendly interface, and interactivity. One, “individual assigned mode” allows instructors to 
assign a question to individual students. The “rush answering mode” encourages students to try to be the first to 
answer the question. Finally, “pushing button mode” is similar to traditional raising the hand to answer. 

In the implementation, the Java Media Framework (JMF) application interface (API) was employed to enable 
audio, video, and other time-based media to be added to the applications. Therefore, a multimedia capturing 
device in the JMF API can act as a source of multimedia data delivery. For example, a microphone, capable of 
capturing raw audio input, and a digital video capture board might deliver digital video from a camera. Such 
capture devices are abstracted as data sources to provide timely delivery of data. The audio/video conferencing 
board is used to deliver both an audio and a video stream. The JMF enables the playback and transmission of real 
time protocol (RTP) streaming through the APIs defined in the javax.media.rtp, javax.media.rtp.event, and 
javax.media.rtp.rtcp packages. The JMF can be extended to support additional RTP-specific formats. We have 
implemented a video conferencing application to capture live audio and video. The system can transmit them 
across the Internet using the RTP API. The JMF API consists mainly of interfaces that define the behavior and 
interaction of objects used to capture, process, and present time-based media. By the intermediary objects, the 
JMF makes it easy to integrate new implementations of key interfaces that can be used seamlessly with existing 
classes. 

Experimental Results 

We examined the correlations among the study variables. The average inter-item correlation was .543. The 
correlation was within an acceptable range (i.e., less than .80), which indicated that multicollinearity was not 
present among the study variables (Hair, Anderson, Tatham, & Black, 1998). Table 1 illustrates the correlation 
between user characteristic, system characteristic, and TAM. The coefficient of system characteristic for 
perceived ease of use (i.e., .807) is greater than that of user characteristic (i.e., .599), while the coefficient of user 
characteristic for perceived usefulness (i.e., .706) is greater than that of system characteristic (i.e., .578). This 
implies that the effect of external variables of user characteristic on perceived usefulness is more than that of 
system characteristic, while the effect of system characteristic on perceived ease of use is more than that of user 
characteristic. 
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Table 1: The correlation analysis 

TAM 

External variables 


correlation coefficient 

Perceived ease of use 

User characteristic 



.599 


System characteristic 



.807 


Perceived usefulness 



.492 


Attitude toward use 



.651 

Perceived usefulness 

User characteristic 



.706 


System characteristic 



.578 


Attitude toward use 



.617 

Attitude toward use 

User characteristic 



.729 


System characteristic 



.731 

Table 2: 

The correlation analysis of user and system characteristics 



Subjective norm Self-efficacy 


PUT 

Computer playfulness 

.563** 

.188 


.235 

Interface style 

.487** 

.648** 


.518** 

Interactivity 

.694** 

.500** 


.619** 


Note. **: p-value < .01 and *: p-value <.05 


Correlation coefficients are presented in Table 2 to illustrate the relationship between user characteristic and 
system characteristic. Most variables from user and system characteristics have significant correlation between 
each other, except for computer playfulness vs. self-efficacy and PUT. This implies that the relationship between 
user and system characteristics is significant. 


Additionally, the internal reliability of multiple-item scales was measured by computing internal reliability alpha. 
Cronbach’s alpha, .953, was within the acceptance range (i.e., greater than .7). We attempted to expand our 
explanatory ability and statistical efficiency by using structural equation model (SEM) to examine a series of 
dependence relationships. The set of relationships, each with dependent and independent variables, was the basis 
of SEM. Based on our research model, we constructed link relationships in path diagram shown in Figure 1. A 
series of causal relationships in a path diagram was translated into a set of equations for estimation. 

Before evaluating the model, we assessed the overall fit of the model to ensure that it was an adequate 
representation of the entire set of causal relationships. The five measurements of goodness-of-fit were useful by 
absolute, incremental, and parsimonious fit measures (Hair et al., 1998). The first measure of overall model fit 
employed the goodness-of-fit index (GFI). The measurement was a good fit if the GFI value was close to 1 and 
greater than .9. The second measure of overall model fit employed the adjusted goodness-of-fit index (AGFI). 
The measure was good fit if its value was greater than .9. Third measure of absolute fit was the root mean square 
residual. This indicated that the average residual correlation was .004, deemed acceptable given the rather strong 
correlations. The fourth measure of overall model fit employed the root mean square error of approximation 
(RMSEA). The fifth measure of overall model fit employed chi-square. This indicated good fit if the value was 
less than 2. Table 3 shows all the goodness-of-fit measurements for our model. The overall situation is good 
except the chi-square divided by degree of freedom. 


Table 3: The goodness-of-fit measurements 


index 

value 

criterion 

acceptability 

GFI 

.990 

>.9 

acceptable 

AGFI 

.886 

>.9 

acceptable 

RMSR 

.004 

<.l 

acceptable 

RMSEA 

.000 

<.l 

acceptable 

Chi-square 

.044 

<.l 

acceptable 


We used a statistical tool, AMOS 6.0 (Arbuckle, 1995), to identify correlations of variables by the path diagram. 
The obtained coefficients representing the strength of each relationship were calculated by the parameter 
estimation of an ordinary least square method between any two paths. The (3 value can show the strength of 
each relationship, while the p value can illustrate the significance of each independent variable if it is less 
than .05. In our longitudinal design, the collected data of the first point in time are illustrated by the SEM, shown 
in Figure 5, for short-term use, which is the tested result using the system for one week. In the perceived ease of 
use (R 2 = .54), it is with the explanatory ability of 54%. The system characteristic has a significant effect on 
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perceived ease of use ((3= .74 and p=0.000). In the perceived usefulness (R 2 = .73), it is with the explanatory 
ability of 73%. Both user and system characteristics ((3= .38, .64 and p=.006, .000) have a significant effect on 
perceived usefulness. In the attitude toward use (R 2 = .64), it is with the explanatory ability of 64%. The attitude 
toward use has a significant effect on perceived ease of use and perceived usefulness (3= .56, .35 and 
p=.000, .006). Thus, hypotheses H 2 , H 3 , H 4 , H 7 , and H 8 are supported. 

The second set of data collected in the eighteenth week, showing the results of having used the system for 9 
weeks, is illustrated by the SEM figure, as shown in Figure 6, for long-term use. The explanatory abilities for 
perceived ease of use, perceived usefulness, and attitude toward use are 65%, 51%, and 63%, respectively. Based 
on the analysis, user characteristic is positively related to perceived usefulness, while system characteristic is 
positively related to perceived ease of use. Hypotheses H 4 and H 8 are supported. We further analyze the variables 
of user and system characteristics to figure out the individual effects. In the user characteristic, subjective norm 
(P= .70 and p=.000) affects significantly on perceived usefulness. In the system characteristic, computer 
playfulness ((3= .62 and p=.000) has a significant effect on perceived ease of use. In addition, computer 
playfulness ((3= .35 and p=.020) affects significantly the attitude toward use. Short-term use is compared with 
long-term use in Figure5 and Figure 6 to observe the effect of the user and system characteristics. In the user 
characteristic, the effect on perceived usefulness for long-term use is more significant than that for short-term 
use. Subjective norm is an important factor for user characteristic. This implies that the perceived usefulness for 
students has a significant effect on the opinions of most classmates and teachers after long-term use of the 
system. The [3 value changes from .38 to .64. The system characteristic with short-term use has a significant 
effect on perceived usefulness, but that with long-term use has not. This is because students perceived the system 
to be interesting, novel, and useful in the first use. However, after having used the system for nine weeks, they 
do not think the system characteristic affects the usefulness. 



Figure 5. SEM of short-term use based on research model. 
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Figure 6. SEM of long-term use based on research model. 


On the other hand, the effect on perceived ease of use for long-term use is more significant than that for 
short-term use. The p value changes from .74 to .81. The effect of system characteristic on perceived ease of 
use for both is significant, especially on the factor of computer playfulness. 


To evaluate the effectiveness of the web-based CRS, we selected two classes to experiment. The survey was 
completely anonymous. Participants of CRS-based group were continuously taught with the classroom response 
systems from the tenth week to the eighteenth week. The questionnaires were given to the participants of 
CRS-based group twice during the study. Thirty-two questionnaires were obtained from CRS-based group in the 
tenth week, and thirty questionnaires were obtained in the eighteenth week. Those in tradition-based group were 
continuously taught in the conventional way from the first week to the eighteenth week. 


Table 4: The difference between two groups 


groups 

midterm exam 

final exam 


(mean, standard deviation) 

(mean, standard deviation) 

CRS-based group 

(84.32,3.03) 

(57.17, 4.08) 

tradition-based group 

(60.61,4.81) 

(32.17,3.61) 


Table 4 illustrates the difference in effectiveness between CRS-based group and tradition-based group. The 
average grades were normalized by Z distribution. The t-test is shown in equation (1). 


T = 


' ■ 

>4 . 

J 11 i. 


(i) 


where pi, p 2 , Si, and s 2 are the average values and the standard deviations of final exams for CRS-based group 
and tradition-based group, respectively, and ni and n 2 are the number of students for the two groups. In the t-test 
equation (1), p 2 is multiplied by 1.4 because the mean of midterm exam for CRS-based group is 1.4 times that of 
tradition-based group. When T is greater than 1.96, we can conclude that the average values of the two groups 
are significantly different under a Type I error level probability, denoted as significance level (.05). The 
computed value, T = 12.08, by the equation (1), is greater than 1.96. Therefore, the grade of CRS-based group is 
better than that of tradition-based group. Thus, hypothesis H 10 is supported. 


DISCUSSIONS AND CONCLUSIONS 

We have implemented the web-based CRS to help instructors conduct their teaching activities. The system 
consists of course entry component, course management component, test banks component, and material 
management component. It also provides reliable aggregation of students’ responses and easily interpretable 
output in the form of a PI graph, and keeps learning tracks. In addition, three playful modes have been developed 
in the system in order to increase computer playfulness, friendly interface, and interactivity for students. 
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In our empirical study, we have examined the effects of user characteristic and system characteristic on 
perceived usefulness, perceived ease of use, and behavior intention for the system. The findings indicate that 
computer playfulness of system characteristic has a significant effect on perceived ease of use and attitude 
toward use, while the subjective norm of user characteristic has a significant effect on perceived usefulness, thus 
supporting hypotheses H 2 , H 3 , H 4 , H 7 , and H 8 . 

For negative hypothesis H 1? for students using the web-based CRSs the perceived ease of use is no longer useful 
for the system. For hypotheses H 2 and H 3 , previous research has successfully applied TAM in the context of 
general web-based learning systems (Yi et al., 2006). The findings strongly support the appropriateness of using 
TAM to understand the factors that influence attitude toward using the web-based CRS. Students are most likely 
to participate in the web-based CRS learning when they perceive it as a useful tool for improving learning 
performance. Additionally, students are willing to use the system to assist their course work if they find it easy to 
use. The results show that perceived ease of use has a stronger effect on attitude toward use than perceived 
usefulness in the context of web-based CRS. One possible explanation is that the web-based CRS is a relatively 
new approach to learning and perceived ease of use is a basic requirement for respondents. It is extremely 
important to make the system easy to interact with, such as through clear and simple navigation buttons of all the 
pages and personalized information search service, since such measures enhance student perceptions of the ease 
of use. For hypothesis H 4 , many past studies have validated this proposition in a wide variety of settings (Hasan, 
2006; Lewis et al., 2003; Yi & Hwang, 2003). User characteristic has a significant effect on perceived usefulness. 
For negative H 5 , this means that user characteristic, including subjective norm, self-efficacy, and PUT, has not a 
significant effect on perceived ease of use. For negative H 6 , user characteristic also has not a significant effect on 
attitude toward use. For hypotheses H 7 and H 8 , system characteristic, such as computer playfulness, interface 
style, and interactivity, has a significant effect on perceived usefulness and perceived ease of use. Prior research 
(Chunga & Tan, 2004; Dagiene & Futschek, 2008; Hasn & Ahmed, 2007; Siau et al., 2006; Tucker, 2008) in the 
effect of system characteristic also indicated that computer playfulness, interface style, and interactivity had 
significant positive relationships with perceived usefulness and perceived ease of use. For negative H 9 , system 
characteristic has not a significant effect on attitude toward use. For hypothesis H 10 , the learning performance of 
CRS-based group is better than that of tradition-based group. Previous studies (Gok, 2011; Preszler et al., 2007) 
indicated that the benefits of using the CRSs can support teaching and learning within lectures and increase 
learning performance. 

The results of analyzing structural equation model illustrate explanatory ability and statistical efficiency between 
dependent and independent variables. In the empirical study of web-based CRS, the computer experience of 
subjects is positively related to perceived ease of use. On the other hand, computer playfulness from interaction 
between the subjects and the system is positively related to perceived ease of use as well. However, computer 
playfulness is not positively related to the perceived usefulness. This implies that computer playfulness is an 
important factor for the perceived ease of using the web-based CRS. In addition, subjective norm is not 
positively related to perceived ease of use, but it has a direct effect on perceived usefulness. This implies that the 
opinions of classmates and teachers are an important factor for perceived usefulness. 

Although most past studies have been examining possible factors of technology adopting acceptance across 
several fields, such as information systems, education, government information systems, as external variables in 
TAM, few researchers aggregate these variables into two characteristic constructs-user characteristic and system 
characteristic. The advantage with separating two classifications is that it helps us understand the difference 
between two characteristics. For the past studies of CRSs, almost all of the CRS studies were based on typical 
question format that required all learners to select from given options. Beyond traditional CRSs, students are 
asked to indicate whether a statement is true or false, or to select an option. Our proposed system provides three 
playful classroom response modes—the individual assigned mode, the rush answering mode, and the pushing 
button mode—to increase computer playfulness, friendly interface, and interactivity for students. To understand 
the adopting acceptance for both user and system characteristics, the study examines the impact of these 
characteristics for the web-based CRS. The results of the empirical study indicate that the effect of user 
characteristic on perceived usefulness is more than that of system characteristic, while the effect of system 
characteristic on perceived ease of use is more than that of user characteristic. For the perceived usefulness, the 
finding implies that user’s beliefs such as subjective norm, self-efficacy, and PUT play an important role in 
system acceptance, especially on the web-based CRS. For the perceived ease of use, it implies that user’s 
perception of using the system, including computer playfulness, interface style, and interactivity, is more 
important than user’s beliefs. 

In addition, the research design of most studies adopted cross-sectional design to collect data at one point in time. 
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This limits causal inferences because temporal priority is difficult to establish. The questionnaires of our 
experiment in this study are based on the longitudinal design and examined two points in time. It can raise causal 
inferences. Moreover, we selected two classes to experiment and compare with learning performance in order to 
understand the difference between CRS-based and traditional teaching. The result indicates that the grade of 
CRS-based group is better than that of tradition-based group. This implies that the benefit of using the CRSs can 
increase learning performance. 

There are some limitations to our research. First, although the results of SEM analysis have shown the effect of 
user and system characteristics on perceived usefulness, perceived ease of use, and attitude of use, only one 
experimental system was investigated. Therefore, the effects of system characteristics may be weaker than those 
of user characteristics, due to the artificial effect of restriction of range. Second, the generalization of the 
findings may be limited to fewer number of subjects. Although there is no single criterion that dictates the 
necessary sample size, more typical is a ratio of five respondents for each estimated variable. Finally, the grades 
of students are regarded as learning performance. This lacks for subjective learning performance which is 
assessed by students. 

In summary, the effect of user characteristic on perceived usefulness is more than that of system characteristic, 
while the effect of system characteristic on perceived ease of use is more than that of user characteristic. For user 
characteristic, efforts to improve perceived usefulness and perceived ease of use, like training, could be used, 
which will enhance self-efficacy of system users. For system characteristic, it is desirable to forecast user 
acceptance as early as possible in the design process. In an early stage of the systems development process, key 
decisions are made, a small fraction of development costs has been incurred, and greatest flexibility exists to 
modify the design. If sufficient user acceptance tests are performed early in the design, the risk of user rejection 
could be reduced and preventive and predictive measures could be applied to ensure future user acceptance. 
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Appendix A. Questionnaire items 

All of 28 items were measured on a five-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly 
agree). In addition to the study variables, the survey included questions about age, gender, and computer 
experience. 

Perceived ease of use 

1) 1 feel that this system is easy to use. 

2) Learning to use this system would be easy for me. 

3) My interaction with this system is clear and understandable. 

Perceived usefulness 

4) Using this system would enhance my effectiveness in learning this course. 

5) Using this system would improve my understanding for the course. 

6) I feel this system useful. 

Attitude toward using 

7) I dislike the idea of using this system to help me learn the course. (Reversed item) 

8) I have a generally favorable attitude toward using this system. 

9) I believe it is a good idea to use this system for learning the course. 

10) Using this system is a foolish idea. (Reversed item) 

Subjective norm 

11) Most classmates whose opinions I value prefer me to use this system in my learning. 

12) For the learning, my classmates who are important to me think that I should use this system. 

13) For the learning, my teacher thinks that I should use this system. 

Self-efficacy 

14) I am able to use this system. 

15) I have the experience of using Internet. 

16) If my classmates are able to use the system, I am able to use the system as well. 

PUT 

17) If I heard about a new information technology, I would look for ways to experiment with it. 

18) Among my peers, I am usually the first to try out new information technologies. 

19) I like to experiment with new information technologies. 

Computer playfulness 

20) I feel that this system is playful. 

21) Using this system, I enjoy the entertainment. 

22) I spontaneously use this system. 

Interface style 

23) I am able to use this input interface. 

24) This input interface facilitates my interaction with this system. 

25) Using this input interface, I can facilitate to control this system function. 

Interactivity 

26) I feel that this system increases more communication with teacher. 

27) This system raises interaction with teacher. 

28) This system makes me and teacher produce more interaction. 
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